System and method for measuring characteristics of cuttings and fluid front location during drilling operations with computer vision

ABSTRACT

The invention relates to a system and method of for measuring the characteristics and volume of drill cuttings. The system comprises at least one camera operably connected to a processor for recording characteristics of drill cutting particles wherein said processor is configured to perform particle detection, extract features of said particles, or both. The processor is typically configured to initiate, interrupt or inhibit an automated activity based on the particle characteristics. The method comprises acquiring visual data from at least one camera, performing particle detection using said data, extracting feature data of any detected particles, alerting an operator and/or initiating, interrupting, or inhibiting automated activity.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application Ser.No. 62/078,573 filed Nov. 12, 2014. Applicant incorporates by referenceherein Application Ser. No. 62/078,573 in its entirety.

This application additionally claims the benefit of U.S. ProvisionalApplication Ser. No. 62/212,252 filed Aug. 31, 2015. Applicantincorporates by reference herein Application Ser. No. 62/212,252 in itsentirety.

This application additionally claims the benefit of U.S. ProvisionalApplication Ser. No. 62/212,233 filed Aug. 31, 2015. Applicantincorporates by reference herein Application Ser. No. 62/212,233 in itsentirety.

FIELD OF THE INVENTION

The invention relates to systems and methods for measuringcharacteristics and volume of cuttings during drilling operations andlocating the fluid front on a shale shaker with computer vision.

BACKGROUND AND SUMMARY

Modern drilling involves scores of people and multiple inter-connectingactivities. Obtaining real-time information about ongoing operations isof paramount importance for safe, efficient drilling. As a result,modern rigs often have thousands of sensors actively measuring numerousparameters related to rig operation, in addition to information aboutthe down-hole drilling environment.

Despite the multitude of sensors on today's rigs, a significant portionof rig activities and sensing problems remain difficult to measure withclassical instrumentation, and person-in-the-loop sensing is oftenutilized in place of automated sensing.

By applying automated, computer-based video interpretation, continuous,robust, and accurate assessment of many different phenomena can beachieved through pre-existing video data without requiring aperson-in-the-loop. Automated interpretation of video data is known ascomputer vision, and recent advances in computer vision technologieshave led to significantly improved performance across a wide range ofvideo-based sensing tasks. Computer vision can be used to improvesafety, reduce costs and improve efficiency.

As drilling fluid is pumped into the well-bore and back up, it typicallycarries with it solid material known as drilling cuttings. Thesecuttings are typically separated from the drilling fluid on aninstrument known as a shale shaker or shaker table. The process ofseparating the cuttings from the fluid may be difficult to monitor usingclassical instrumentation due to the violent nature of the shakingprocess. Currently the volume of cuttings is difficult to measure andtypically requires man-power to monitor. Knowledge of the total volumeand/or approximate volume of the cuttings coming off the shaker tablemay improve the efficiency, safety, and/or environmental impact of thedrilling process.

Additionally, the location and orientation of the fluid front on theshale shaker is an important parameter to the drilling process that maybe difficult to measure accurately. Currently this is somewhat difficultto measure and requires man-power to monitor.

Particulate matter that is returned up the well-bore during drillingalso contains a great deal of information about the lithology and/orproperties of the subsurface, and can give significant insight into thebehavior of the well-bore (e.g., indicating cave-ins, or failure toclean). Current drilling operations require human-in-the-loop analysisof these cuttings; a geologist has to go inspect the cuttings on aconveyor belt or other receptacle down-stream from the shale-shakers.This process is time consuming, expensive, and error prone. Classicalinstrumentation approaches to particle analysis are extremely difficultto design and implement—the sizes, shapes, and consistencies of cuttingsprohibit most automated mechanical handling and measurement. Incontrast, significant information can be obtained from visual analysisof the particles on the shaker and this information can be used to makebetter decisions about proper drilling parameters quickly.

Therefore there is a need for an automated computer vision basedtechnique for identifying cuttings on a belt, and estimating variousfeatures regarding their shape, size, volume and other parameters.Information from this system can be used to provide real-timeinformation about the well-bore to the drill-team, flag unexpectedchanges in the particle sizes and shapes, and/or provide a long-termrecording of the particle characteristics for post-drilling analyses.

There is also a need for an automated computer vision based techniquefor estimating the location of the fluid front on the shale shaker.

This information may also be used to optimize, improve, or adjust theshale-shaker angle (saving mud, and/or increasing efficiency); alert anoperator to expected and/or unexpected changes in the cuttings volumeswhich may, in some cases, be indicative of hole cleaning, influx,losses, and/or other problems; and show whether or not the volume andcharacteristics of cuttings exiting the shaker is less than, greaterthan or approximately commensurate with the rate of penetration (“ROP”).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts one of many potential embodiments of a system involvingat least one camera and processor for monitoring drilling cuttings.

FIG. 2 depicts an embodiment in the context of a simplified wellcirculatory system and multiple sensors which may assist the system indetermining the volume and/or characteristics of drill cuttings.

FIG. 3 depicts a block diagram showing the steps of a potential methodfor measuring the characteristics of drill cuttings.

FIG. 4 depicts a block diagram showing the steps of a potential methodfor localizing a fluid front on a shaker table.

DETAILED DESCRIPTION

The disclosed method and system typically contains several partsincluding at least one camera 102 (video, or single-frame) oriented asto view a shaker table 206 on which the particulate matter passes and/ororiented to view the cuttings 104 as they approach and/or fall off theedge of a shaker table 206. The system and variations of the system mayalso include a belt 204, shaker table screen 208, machinery controlsystem 116 and other drilling or industrial equipment 210. Cameras 102may be oriented as to provide off-axis views (e.g., 90 degrees offset),or may be oriented in the same general direction, but spatially offsetto provide stereo vision. In alternative embodiments, Red GreenBlue-Depth (“RGB-D”) cameras, ranging cameras, and/or otherdistance-sensing technologies 103, such as Light Detection and Ranging(“LIDAR”), may be used in addition to, or in place of cameras 102.

Depending on the speed of the belt 204 and the rate at which particles104 are moving, cameras may collect frames as slow as 0.003 Hz (1frame/5 minutes) or much faster. Each camera 102 may comprise or beconnected to a computer 110 which performs particle 104 detection and/orextracts one or more features (e.g., statistical descriptors, RGBvalues, texture features, edge-descriptors, object matching descriptorsor bounding boxes) from one or more up to each detected particle 104.

Information about these particles 104 may be accumulated on a centralcomputing resource 110. In the case of multiple cameras 102, informationabout the camera's 102 relative pose and orientation, and thecorresponding particle 104 bounding boxes may be used to combineinformation about particles 104 that may be visible in both cameras 102.The resulting information about particles 104 may then be tracked overtime and logged to a database 112 for later retrieval and furtheranalysis. Alternatively, this information may be tracked over time andchanges in the statistical distributions of the particles 104 may beflagged and brought to the mud-logger's or drill-team's attention with,for example, a plain-text description of the observed change (e.g., “theaverage cutting size has increased by 20% in the last 3 minutes”), andthe corresponding video data. This information could also be used in asupervised classification system, trained using prior data to identifyspecific discrete cases—e.g., “cutting is from a cave-in”, or “cuttingis due to X”. Supervised classification may be used on each particle 104independently, or on the statistics of the particles 104 in the currentframe and recent time in aggregate. Outputs from the classificationsystem may be presented to the mud-logger's or drill-team's attentionwith, for example, a plain-text description of the observed change(e.g., “cuttings indicative of cave-in detected”), and the correspondingvideo data.

Each camera 102 may comprise, or may be connected to, a processor 110which may be configured to perform detection and localization of thedrilling cuttings 104 on the shaker 206. The processor 110 mayadditionally or alternatively be configured to identify cuttings 104,track the cuttings 104, and/or estimate the volume of cuttings 104coming off the end of a shaker 206. These actions may also be performedon a per unit time basis when desirable. In some embodiments,information from a camera 102 may be combined with information frommultiple other sensors 220. Information related to the flow-in, drillingpumps, flow-out, and/or pit volume, collectively known as thecirculation system 222, may be useful in combination with someembodiments. By combining this information, the system may be able toprovide different information and/or alerts under different conditions,such as when the pumps are on vs. off Information across the differentsensor 220 modalities may be fused to allow the system to make betterdecisions under certain circumstances.

Disclosed embodiments include many possible combinations of cameras 102,distance-sensing equipment 103 and sensors 220. For example, optical orvideo cameras 102, single or multi-stereo-cameras 102, night-visioncameras 102, IR, LIDAR, RGB-D cameras, or other recording and/ordistance-sensing equipment 103 may all be used, either alone or incombination. Each camera 102 or combination of cameras 102 and sensors220 may be used to track the volume or other characteristics of cuttings104 on or exiting a shaker 206. Information from the cameras 102 and/orsensors 220 may be combined with information from the circulation system222 (e.g., flow-in, flow-out, and pit-volume) to modify the system'sbehavior as desired.

Information about the absolute and/or relative change in cutting 104volumes or other characteristics of cuttings 104 coming off of theshaker table 206 may, under certain conditions, be combined withcirculation system 222 parameters and/or other drilling parameters, suchas rate of penetration, and be relayed to the drilling engineer or otherpersonnel. For example, a sudden change, either decreases or increases,in the cuttings 104 volume not correlated to changing rate ofpenetration may indicate hole cleaning problems, influxes, and/or otherchanges in conditions. Additionally, a sudden change in the spatialcharacteristics of the cuttings 104 may indicate a cave-in or otherphenomena.

Cameras 102 (optical, IR, RGB-D, single, stereo, or multi-stereo amongothers) may be mounted within pre-defined constraints around the shakertable 206. In an embodiment, camera 102 orientations are approximately45 degrees to the shaker table 206, but cameras 102 may be placedanywhere with a view of the cuttings 104 and/or the fluid front 118.This may include from 0 to 180 degrees pitch. When using a single camera102, it may be preferable to place the camera 102 within a range of 60degrees to −60 degrees of vertical. The camera 102 may be configured tocapture a view from above, oriented approximately down at the top of theshaker 206.

In some embodiments, multiple cameras 102 may be placed in mutuallybeneficial locations. As an example, stereo vision approaches mayimprove particle 104 size estimation. Stereo cameras 102 typically viewthe same scene from approximately the same angle but from differentspatial locations. Alternatively, cameras 102 viewing the same scenefrom different angles, such as a front view, side angle view, and/oroverhead view may provide different views of the same objects and mayreduce the need for assumptions, such as rotational symmetry, in volumeor other characteristic estimation. Additionally, when using multiplecameras 102, the preferred placement may be a function of the shapeand/or size of the shaker 206, the desired volume or characteristicfidelity, and/or the configuration of sensors 220 under consideration.Preferably, multiple camera 102 placements may be configured to provideadditional information from each camera 102 or sensor 220 as discussed.

Cameras 102 may be equipped with a flash or other light source 106 tomaintain substantially adequate illumination across multiple images.This may be useful since the ambient lighting can change significantlydepending on the time of day or night and/or the weather conditions. Bymaintaining adequate lighting, some processing complications may be ableto be avoided.

In some embodiments, cameras 102 and/or distance-sensing equipment 103may be configured to move in response to pre-determined criteria. Thismovement may comprise rotation, panning, tilting and/or zoom adjustmentsalong any axis. The movement may be automated or may be performed bystaff. These adjustments may be predicated on conditions including butnot limited to observed features or characteristics of the cuttings,environmental conditions surrounding the rig and/or shale shaker and/orinput from other sensors.

Different behaviors of the cuttings 104 and shakers 206 may be expectedduring active-flow periods when the mud pumps 224 are running andpassive periods when the mud pumps 224 are off Additional changes maymanifest during the transient periods shortly after the pumps 224 switcheither on or off Additional data about the drilling process, such ashook load, bit depth, or rate of penetration, among others, may also beused to provide contextual information to the computer vision system incertain conditions.

In some embodiments, discrete cuttings 104 may be identified on or nearthe shaker 206, and/or as they fall off the end of the shaker 206 usingone of many image processing features and techniques. Backgroundsubtraction and/or change detection may be used to identify cuttings 104in part because cuttings may appear different than the typicalbackground, which may consist of a shale shaker 206, shale shaker screen208, and/or other background features. Cuttings 104 may also appeardifferent from the typical background when falling off the edge of theshaker 206. Cuttings may additionally appear to “move” at anapproximately constant velocity across a shaker table 206. Thesefeatures may enable background estimation and/or subtraction techniquesto be used to identify individual cuttings 104. Texture features mayalso be used for detection of drilling cuttings 104. Cuttings 104 mayhave an image texture which is different from the background. This mayallow the cuttings 104 to be detected using this difference in texture.This detection may be accomplished using one-class classifiers todistinguish cuttings 104 as differences from the background and/orvice-versa. Two-class classifiers may also be used to activelydistinguish two classes, one class for cuttings 104 and another forbackground. It will be appreciated that multiple-class classifiers mayalso be used when desirable.

In other embodiments, reflectivity and/or color properties may also beused for cutting 104 detection. Cuttings 104 may often be covered indrilling fluid and therefore may have different reflectivity and/orcoloration than the background (shale shaker 206, conveyor belt 204,and/or other background features). Cuttings 104 may therefore bedetectable using these changes in color and reflectivity. It will benoted that these techniques may also be applicable when the cuttings 104are not covered in drilling fluid, as long as the cuttings 104 do notpresent the same reflectivity and color characteristics as thebackground.

Alternative embodiments may additionally and/or alternatively usepersistence and/or tracking techniques to identify cuttings 104.Cuttings 104 often maintain approximately constant shape and size asthey travel across the shaker 206. As a result, individual cuttings 104may be able to be tracked and/or disambiguated across multiple frames.Tracking cuttings 104 may be accomplished using any of a number oftracking techniques, (e.g., Kalman filters, particle filters, and/orother ad-hoc tracking techniques). This may enable resolution of thecuttings 104 as multiple “looks” are aggregated on each cutting 104. Insome embodiments, this may enable more accurate volume or characteristicestimation as well.

Still more embodiments may use fluid and/or cuttings 104 velocityestimation to identify cuttings 104. Cuttings 104 often move across theshaker screen 208 at approximately the same velocity as one another.This velocity may be estimated across all of the observed cuttings 104and/or be tracked (e.g., with a Kalman filter or particle filters). Thisinformation may then be used to identify other cuttings 104 and/orpredict the eventual locations of cuttings 104 that may be temporarilylost during the tracking and identification stage. Changes in thisvelocity may also be flagged to an operator.

In embodiments that comprise multiple cameras 102, LIDAR, RGB-D camerasand/or other distance sensing equipment 103, particles 104 may beidentified using the observed “height” of the cuttings 104 as comparedto the expected background height.

Techniques similar to those discussed may also be applicable inhyperspectral, IR, or other imaging modalities. As cuttings 104 aretracked on the shaker 206, conveyor belt 204, and/or other devices,their volume and characteristics can be estimated in several ways. Inembodiments using single-sensor RGB cameras 102 or similar devices, theapproximate volume of cuttings 104 may be estimated from a singleviewpoint using rotationally symmetric assumptions about the cuttings104, and the known, calculated, and/or estimated camera-to-shaker tabledistance. Alternatively, a cutting 104 shape inference may be made usingknowledge of the available light source and estimating the reflectivityas a function of space on the visible parts of the cutting 104.

For embodiments using single-sensor RGB cameras 102 or similar devices,the approximate volume of the cuttings 104 may also be estimated usingpreviously trained regression techniques which determine the approximateobject volume using image region features (e.g., eccentricity, perimeterlength, and area among others) extracted from individual cuttings 104 inthe image. These image region features may be used to identify changesin the cutting 104 shapes as well.

Embodiments which use multiple cameras 102, combined camera 102 (e.g.,stereo-camera) scenarios, or distance detection sensors 103,depth-information may be directly available and/or inferable. This mayprovide the visible cross-section of the object and/or a measure of howthat cross-section varies with height. This information may be used toimprove the volume estimation by reducing the symmetry assumptionsrequired to estimate the volume of each cutting 104.

In some embodiments, the total volume of all the cuttings 104 visible ina scene, image, and/or frame may be estimated by combining informationfrom the detection, tracking, and/or volume estimation portions of thetechniques discussed. In other embodiments, the net volume flow may becalculated by considering the amount of volume entering or exiting thevisible region per unit time. Alternatively, the change in volume may beestimated by calculating the volume of particles 104 passing by aspecified “line” in real-world space (e.g., the end of the shaker 206),or through a specified region on the shaker 206 or in the background.Depending on the particular installation, camera 102 availability,and/or configuration, the total volume estimation may be appropriate foractual volume estimation in real-world units (e.g., 1 m³ of cuttings 104per 5 minutes), and/or in relative terms (e.g., a 5% increase incuttings 104 volume in the last 5 minutes). Both may be valuable metricsin certain circumstances, but real-world units are preferable as thepercent change can be derived from this information.

In still more alternative embodiments, information from the camera(s)102 may be combined with information from the circulation system 222(e.g., flow-in, flow-out, ROP, and/or pit-volume) or other rig sensors220 to change the detection system behavior. As discussed, informationacross the different sensor 220 modalities may be fused to make betterdecisions. As drilling continues, the camera 102 system may be able toauto-calibrate to determine what a realistic amount of cuttings 104 permeter drilled is (e.g., leveraging ROP), and may additionally use thisfor automatic alarming if the observed volume of cuttings differs ordiverges significantly. In addition to activating an alarm 114, thesystem may initiate, alter, interrupt, or inhibit automated activity byequipment 210 connected to the system.

Information regarding sudden unexpected changes in the volume, shapes,velocities, and or other characteristics of the cuttings 104 can bebrought to the user's attention visually, audibly, or with othernotifications. These notifications may be complete with photographs ofthe current situation and/or a plain-text description of the cause ofthe alarm (e.g., “sudden increase in volume of cuttings”).

In other embodiments, the video data and/or other data may also betagged along with any information extracted during the computer visionprocessing process. Gathered information may be displayed to an operatorwith a user interface which may include an annotated image of the shakertables 206 under consideration. This image may be automaticallyannotated and may also, in certain embodiments, display marksidentifying a variety of key features, such as the fluid front 118,cuttings 104, any potential issues, etc.

In another embodiment, the volume of cuttings 104 on or coming off theshaker table 206 may be estimated using a two-step process of particle104 detection followed by volume estimation. The use of RGB and IRcameras 102 may be useful under certain circumstances. Particle 104detection can be accomplished using any of a number of image processingtechniques, including but not limited to corner detection, blobdetection, edge detection, background subtraction, motion detection,direct object detection, adaptive modeling, statistical descriptors anda variety of similar techniques. The proper particle 104 detectionapproach may be site-dependent, based on the local lithology. Objectdetection may also be obtained using standard background depthestimation and/or subtraction approaches. The use of distancingequipment 103, such as LIDAR and/or RGB-D cameras, may have advantageswith regard to these techniques.

Once a cutting 104 has been detected, a camera 102 may be used toestimate cutting 104 volumes or other characteristics using the knowncamera 102 transform parameters, the known distance to the shaker 206,and/or the shape and/or size of the detected object as it appears in thecamera 102 frame. Similar processing is applicable for many types ofcameras 102, such as RGB, and IR cameras 102. For multiple cameras 102viewing the same scene, stereo vision techniques may be used to obtain apotentially more detailed 3-D representation of the cuttings 104, andthereby achieve more accurate volume estimations. If RGB-D or LIDAR 103data is available, these may be used to render 3-D models of thecuttings 104, for higher fidelity volume estimation.

Given the particle's 104 bounding boxes, various features about eachdetected particle 104 may then be extracted. These include variousobject shape parameters (e.g., image moments), texture features, HOGfeatures, color descriptors, integral channel features, or the raw pixeldata.

Information about particles 104 may be aggregated both temporally andspatially. Spatial information can be accumulated across multiplecameras 102 by using information about each individual camera's 102 poseto infer when detections in the two cameras 102 correspond to the samecutting 104. Similarly, information can also be aggregated temporallywhen the camera 102 frame-rate is fast enough to capture multiple imagesof the same object as it passes down the belt 204 in the same camera102.

After aggregation, one or more up to each unique detected particle 104,but typically a representative sample size, is associated with acorresponding feature vector (an accumulation of the image moments, RGBvalues, HOG features, etc.). These features can be used to performinference, track statistics, and perform classification as discussedbelow. In some embodiments, all or most of the available particles 104will be detected by the system, while many other embodiments will detecta significant percentage, but not 100% of all particles 104. Certaindisclosed embodiments may be able to function as described with a verysmall but representative sampling of the total number of availableparticles 104. A representative sample may be as few as 0.1% of allavailable particles 104. Alternatively a representative sample may begreater than 1%, greater than 10%, greater than 25%, greater than 50%,greater than 75%, greater than 90% or greater than 95% of the totalparticles 104 available.

The information about each cutting 104 as it passes through the camera102 field-of-view, along with the complete image data, cutting 104bounding boxes, associated features, and meta-data (e.g., time, rigidentifier, weather information, drill bit depth, etc.) may be recordedto a database 112 which can be leveraged post-hoc and combined withdatabases 112 from other drilling experiments.

The statistical distribution of the cuttings 104 may be tracked as afunction of time. This may be accomplished using either parametric(e.g., Gaussian distributions) or non-parametric Bayesian models (e.g.,Dirichlet process mixtures) or using adaptive histograms, where thecurrent density, p(x, f), is estimated using:

${p\left( {x,f} \right)} \propto {\frac{\tau\;{h\left( {x,f} \right)}}{N_{t}} + {\left( {1 - \tau} \right){p\left( {x,{f - 1}} \right)}}}$

Where N_(t) represents the number of cuttings 104 detected in the framenumber f, h(x, f) represents the histogram of features (x) from all Ncuttings 104 in the current frame, τ∈[0,1] controls the speed ofadaptation, and p(x, f−1) represents the density estimate from theprevious frame.

When the likelihood of the current data, x, is very low, e.g.,p(x,f−1)<θ

This indicates a sudden change in the distribution of the particles 104,which should be brought to the attention of the mud-loggers ordrill-team.

Supervised classification and regression techniques may be utilized toautomatically flag any of a number of discrete events that result inchanges to the statistics of the cutting 104 features. Supervisedtechniques rely on a body of previous data from similar scenarios withcorresponding labels (e.g., videos of cuttings 104 from cave-inscenarios, videos of cuttings 104 when the well-bore is not beingcleaned adequately, etc.). Given this historical data, features may beextracted from the historical video data and supervised techniques(e.g., SVM, RVM, Random Forest, linear discriminant analysis, quadraticdiscriminant analysis) may be trained to identify these scenarios in newdata as it is collected. When new data is collected, the outputs fromthese supervised classifiers are then presented to the mud-loggers ordrill-team as appropriate.

Information regarding sudden changes in the statistical distributions ofthe cuttings 104, as well as flags raised by the supervised techniquesdescribed above may require immediate attention on the part of themud-loggers or drill-team. Depending on the severity of the type ofchange encountered, information from the system may be presented to thecorresponding person in one of several ways. Information may be enteredin the daily automated mud-log (least importance). Information may alsobe conveyed to appropriate personnel via e-mail, text message, orsoftware-pop-up on the driller's screen, etc. (moderate importance).Alarms 114 or other communications requiring immediate response may alsobe raised (most importance).

In some cases, alarms 114 may contain a clear text description of theissue discovered (e.g., “The average cutting size increased by 20% inthe last 3 minutes”). This alarm 114 may be provided together with avisualization of the camera 102 data prior to the alarm 114, the camera102 data that caused the alarm 114, diagnostic text, and arrowsillustrating the observed changes.

If the processor 110 detects a change in particle 104 characteristics ordetects a per-determined condition, the processor 110 may initiate,interrupt, alter or inhibit an automated activity using a machinerycontrol system 116. The machinery control system 116 may increase ordecrease the speed of a belt 204. The machinery control system 116 mayadjust the tilt of a shale shaker 206 or may make adjustments to thefunctioning of any other piece of equipment 210. Equipment 210 mayinclude but is not limited to all forms of shake shakers 206, shakerscreens 208, drill bits, drilling motors, top drives, pipe elevators,mud pumps 224, valves, and a wide array of other drilling and industrialequipment.

Various control mechanisms may be appropriate to automate the angleand/or position of the shale shaker 206. For example, PID controllersand/or other systems may be used to adjust the shaker 206 based onacquired data. These adjustments may be done automatically, via aclosed-loop system, or by instructing an operator to make the necessarychanges based on the acquired data.

The cameras 102, distance sensing equipment 103 and/or other sensors 220and techniques discussed above may additionally be used to identify andlocalize the fluid front 118 on a shaker table 206. The fluid front 118is typically where the majority of the mud and/or cuttings slurry ends.This may be where the separated shale cuttings 104 begin and/or wherethe shaker screen 208 is exposed. The information related to the fluidfront 118 may be tracked over time and logged to a database 112 forlater retrieval and/or further analysis. This information may also betracked over time and changes in location or behavior of the fluid maybe brought to the mud-logger's or drill-team's attention using anysuitable technique, such as a plain-text description of the observedchange (e.g., “the fluid front appears to be too far forward on theshaker table”). The corresponding video data may also be provided to thedrill-team to allow for independent verification of the alertconditions. The fluid front 118 information may also be used in aclosed-loop control system to adjust various parameters, such as theangle or speed of the shaker table 206, if desired. Some embodiments ofthis system may allow the adjustments to be made automatically withouthuman involvement.

Disclosed embodiments include many possible combinations of cameras 102,distance sensing equipment 103 and sensors 220. Each camera 102 orcombination of cameras 102 and sensors 220 may also be used to track thelocation of the fluid front. Information from the cameras 102 and orsensors 220 can be combined with information from the circulation system222 (e.g., flow-in, flow-out, and pit-volume) to modify the system'sbehavior as desired.

Different behaviors of the fluid front 118 and/or shakers 206 may beexpected during active-flow periods when the mud pumps 224 are runningand passive periods when the mud pumps 224 are off Additional changesmay manifest during the transient periods shortly after the pumps 224switch either on or off Additional data about the drilling process, suchas hook load, bit depth, or rate of penetration, among others, may alsobe used to provide contextual information to the computer vision systemin certain conditions.

In some embodiments, the fluid front 118 may be identified using acomputer vision and machine learning system. For example, textureclassification may potentially be used to identify the fluid front 118since the visual “texture” of the mud as the shale-shaker 206 isvibrating is typically different from the visual texture of the shakertable 206 and/or other nearby objects. The visual texture of thesplashing, vibrating, and/or moving fluid behind the fluid front 118stands in contrast to the relatively regular texture of the rest of theshaker 206. As a result, it may be possible to detect the fluid front118 using texture features. These features may be used to distinguish anarea from the shaker table 206 and/or background features, (e.g., sincethe distinguished area differs from the shaker 206 and/or background),and/or used in a multi-class classification framework (e.g., a 2-classsupport vector machine (“SVM”)) to distinguish the “shaker” and/or“background” class from the “fluid” class.

Another example of computer vision that may be used to identify thefluid front 118 is change detection. The shale shaker 206 itself mayprovide a relatively static background. Even when the shaker 206 ismoving, the information related to the pixels viewing the shaker 206 mayremain stationary. In some embodiments, the information related to thepixels may include the statistical distribution of the pixel intensitiesin any color space (e.g., RGB). This may allow long-term backgroundestimation (e.g., via Gaussian Mixture Models, robust principalcomponent analysis, etc.) to be used to estimate the background classwhen the pumps 224 are off and/or shortly after the pumps 224 turn onand before fluid appears on the shaker 206. This technique may also beused when the pumps 224 are on under certain conditions. These modelsmay also be used to flag changes, which may be caused by the advent ofthe fluid front on the shaker 206.

An additional example of computer vision that may be used to identifythe fluid front 118 is reflectivity and/or color detection. The fluidfront 118 is often a different color than the shale shaker 206 and mayhave different reflectance characteristics as well. Reflectance and/orcolor features may be used for fluid and/or shaker classification ontheir own, in combination with each other, and/or in combination withother disclosed techniques. Additionally, numerous other descriptorvectors may also be used in conjunction with and/or in addition to thetechniques disclosed above. Other possible techniques include, but arenot limited to, histogram of oriented gradients (“HOG”), scale invariantfeature transform (“SIFT”), speeded-up-robust-features (“SURF”), binaryrobust independent elementary features (“BRIEF”), Viola-Jones, (“V-J”),Harr wavelet, (deep) convolutional neural networks (CNNs) and others.

Detection of the actual fluid front 118, as compared to the other fluidregions may be accomplished by classifying regions of the image as“fluid,” “non-fluid,” or any other classifier, and/or using image regionproperties on the resulting image regions to determine a potential classseparating line. The fluid front 118 may be specified as a line, as amore complicated smoothly varying function (e.g., spline, quadratic,etc.), and/or as a combination of any of these. Preferably, the frontshould be constrained to be on the shale shaker 206.

In some alternative embodiments, the image may be separated into fluidand/or non-fluid regions by hypothesizing numerous locations for thefluid front 118 and choosing at least one location corresponding to alocation above a certain threshold likelihood separation between twodistributions, one representing the fluid and one representing theshaker. Preferably, a chosen location corresponds to the maximumlikelihood separation between the two distributions.

Still other optimization techniques may also be used to identify thelocation of the fluid front 118. For example purposes only, geneticalgorithms, Markov-chain monte-carlo (“MCMC”) techniques, additionalbackground subtraction and/or correction techniques, among many othertechniques may all be used.

Once detected, the fluid front 118 location (line, quadratic, splineformulation, and/or any other demarcation) may be tracked over time.This may be accomplished through many different techniques. For examplepurposes only, tracking the fluid front 118 over time may beaccomplished by appropriately parameterizing the fluid front 118representation and leveraging time-sensitive Kalman or ParticleFiltering approaches to update the location of the fluid front 118 in aframe. Preferably this would be done in many frames, and most preferablyin every frame. Alternatively, the fluid front 118 location may bere-estimated in one, some, and/or all frames. This may also be done whenthe fluid front 118 was not previously detected.

In some embodiments, the fluid front 118 location may be logged to adatabase 112 for later retrieval and further analysis. Changes in thelocation or behavior of the fluid may be brought to the mud-logger's ordrill-team's attention with a plain-text description of the observedchange (e.g., “the fluid front appears to be too far forward on theshaker table”), and the corresponding video data.

In other embodiments, the video data and/or other data may also betagged along with any information extracted during the computer visionprocessing process. Gathered information may be displayed to an operatorwith a user interface which may include an annotated image of the shakertables 206 under consideration. This image may be automaticallyannotated and may also, in certain embodiments, display marksidentifying a variety of key features, such as the fluid front 118,cuttings 104, any potential issues, etc.

Various control mechanisms may be appropriate to adjust and/or automatethe angle and/or position of the shale shaker 206. For example, PIDcontrollers, hydraulic pistons, electronic motors, and/or other systemsmay be used to adjust the shaker 206 based on acquired data.

The fluid-front 118 location estimation may also be used in aclosed-loop control system to adjust various parameters (e.g., theangle) of the shaker table 206. This may enable automatic control of ashaker table 206 system on a rig. This may result in saving time, savingmoney, and/or preventing undesirable ecological impacts.

FIG. 1 depicts one of many preferred embodiments of the system. Cuttings104 on a shaker 206 pass through the fluid front 118 and may beilluminated by a light source 106. Cameras 102 and distance sensingequipment 103 are configured to gather data related to thecharacteristics of the cuttings 104. The cameras 102 and distancesensing equipment 103 are connected to a processor 110 which is operablyconnected to a database 112, alarm 114 and a machinery control system116. The machinery control system 116 is configured to initiate,interrupt or inhibit automated activity by equipment 210. The processor110 may also be connected to a display 120 in order to provideinformation to an operator.

FIG. 2 depicts a very simplified oil well circulation system 222 whichmay contain many sensors 220 in addition to mud pumps 224 and drillingequipment 210. It will be appreciated that the specific configuration ofthe well circulation system 222, sensors 220, mud pumps 224 and drillingequipment 210 may be very different in alternate embodiments disclosedherein. Shale shaker 206 is generally in the path of drilling mudcirculation and may be used to screen out cuttings 104 for analysis aswell as to preserve drilling mud.

FIG. 3 shows a block diagram of the steps for a certain method ofmeasuring the characteristics of drill cuttings. The disclosed steps maybe organized in a different manner. In certain embodiments one or moreof the steps may be removed or exchanged.

FIG. 4 shows a block diagram of the steps for a certain method oflocalizing the fluid front 118 on a shaker table 206. The disclosedsteps may be organized in a different manner. In certain embodiments oneor more of the steps may be removed or exchanged.

Disclosed embodiments relate to a system for monitoring characteristicsof drilling cuttings 104 and adjusting automated activity based on thecharacteristics. The system includes a shaker table 206 which isoperably connected to a machinery control system 116. The system alsoincludes at least one camera 102 operably connected to a processor 110.The processor 110 may be configured to perform particle detection,extract features of the particles 104, estimate the volume of theparticles 104 using machine vision or a combination of all three steps.The processor 110 is also operably connected to the machinery controlsystem 116 which is configured to adjust automated equipment 210 basedon input from the processor 110.

Certain embodiments of the system may also include distance sensingequipment 103 and sensors 220 for detecting pre-determined parameters ofa well circulatory system. The sensors 220 may include a well flow-insensor 226, flow-out sensor 228 and/or pit volume sensor 230. The systemmay also include a light source 106 configured to provide lightingduring diverse weather conditions and times of day.

Additional embodiments of the system may include the speed and/or thespeed and/or angle of the shaker table 206 being adjusted based oninformation received from the processor 110. Certain embodiments of thesystem may also include at least two cameras 102 which are configured toprovide stereo vision. Other embodiments may additionally oralternatively include at least two cameras 102 configured to monitor theshaker table 206 from significantly different angles.

Some embodiments of the system may include a bit-depth sensor 232 and/ora database 112 for recording particle information. Certain embodimentsmay include a database 112 which is capable of comparing current dataagainst historical data. The system may also include an alarm 114 foralerting staff or an operator to the occurrence of a pre-determinedcondition.

Certain embodiments related to a method for measuring thecharacteristics of drill cuttings. The method may include the steps ofacquiring visual data 302 from at least one camera, compiling visualdata from multiple sources 304, performing particle detection 306 usingthe data and a processor 110, extracting feature data 308 of anydetected particles, recording the visual or feature data 310 for futurereference, comparing the visual or feature data against a database ofpreviously recorded data 312, displaying the visual or feature data 314to staff and/or initiating or interrupting automated activity 318 usinga machinery control system operably connected to the processor based onthe extracted feature data.

Additional embodiments may relate to a system for monitoringcharacteristics of drilling cuttings exiting a shaker table. The systemmay include a shaker table screen 208, at least one camera 102 ordistance sensing equipment 103 configured to monitor the shaker tablescreen 208. The camera 102 or distance sensing equipment 103 may beoperably connected to a processor 110. The processor 110 may beconfigured to identify drill cuttings 104 and estimate the volume and/orcharacteristics of the cuttings 104 on the screen 208 using machinevision techniques.

Disclosed embodiments relate to a method for detecting or localizing afluid front 118 on a shaker table 206. The method may include the stepsof acquiring visual data 402 using at least one camera, compiling visualdata from multiple sources 404, performing fluid front localization 406using the data and a processor, recording data 408, comparing dataagainst a database of previously recorded data 410, displaying data 412and initiating, altering or interrupting automated activity 414 using amachinery control system operably connected to the processor based onthe fluid front localization.

What is claimed is:
 1. A system comprising: a shaker table; at least onecamera operably connected to a processor, the processor configured toperform particle detection, wherein the processor extracts features ofthe particles, estimates the volume of the particles, estimates theshape of the particles, or a combination of all three using machinevision, wherein the machine vision comprises analyzing a statisticaldistribution of data tracked as a function of time; wherein the systemis configured for localizing the location of a fluid front on a shakertable and wherein the processor is operably connected to the shakertable and configured to adjust automated activity of the shaker tablebased on the localized location of the fluid front wherein localizingcomprises detecting or estimating the actual fluid front location by theprocessor.
 2. The system of claim 1, configured for monitoringcharacteristics of drilling cuttings and adjusting automated activity ofthe shaker table based on the characteristics.
 3. The system of claim 1further comprising distance sensing equipment operably connected to theprocessor for sensing the distance from the camera to the shaker table.4. The system of claim 1, further comprising at least one sensor fordetecting a predetermined parameter of a well circulatory system.
 5. Thesystem of claim 1, further comprising a well flow-in sensor, flow-outsensor, and pit volume sensor.
 6. The system of claim 1, furthercomprising a light source arranged and designed to provide lightingduring diverse weather conditions and times of day.
 7. The system ofclaim 1, wherein the speed and/or angle of the shaker table may beadjusted based on information received from the processor.
 8. The systemof claim 1, further comprising at least two cameras configured toprovide stereo vision.
 9. The system of claim 1, further comprising atleast two cameras configured to monitor the shaker table fromsignificantly different angles.
 10. The system of claim 1, furthercomprising a bit-depth sensor.
 11. The system of claim 1 furthercomprising a database.
 12. The system of claim 11 wherein said databaseis capable of comparing current data against historical data.
 13. Thesystem of claim 1 further comprising an alarm system for alerting staffto the occurrence of a pre-determined condition.
 14. The system of claim1 wherein the processor estimates the volume of the particles andestimates the shape of the particles.
 15. A method for measuring thecharacteristics of drill cuttings comprising: acquiring visual data fromat least one camera; performing particle detection using said data and aprocessor; extracting feature data of a representative sample ofdetected particles using machine vision, wherein the machine visioncomprises analyzing a statistical distribution of data tracked as afunction of time; detecting changes in the feature data of the detectedparticles; and localizing the location of a fluid front on a shakertable by a processor operably connected to the shaker table andconfigured to adjust automated activity of the shaker table based on thelocalized location of the fluid front wherein localizing comprisesdetecting or estimating the actual fluid front location by theprocessor.
 16. The method of claim 15, further comprising comparing saidvisual or feature data against a database of previously recorded data.17. The method of claim 15, further comprising compiling visual datafrom multiple cameras and performing particle detection on the compiledvisual data.
 18. The method of claim 15, further comprising alertingstaff to the occurrence of a pre-determined condition.